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BY PRADEEP KOTAMRAJU 

C areer and technical educa¬ 
tion (CTE) is increasingly 
being seen as a major 
contributor to the recovery 
of Lhc U.S. economy. However, questions 
exist as to the e flee live ness and impact of 
CTE. One is whether the federal invest¬ 
ment (Carl D. Perkins Act) in CTE is 
paying off. To answer this, we need to 
establish the internal efficiency of CTE by 
comparing the costs and benefits of imple¬ 
menting CTE at the local or state levels. 

A second question is whether CTE has a 
measurable impact. This question focuses 
on external effectiveness. 

Answering these questions may put 
to rest the frequently held notion that 
CTE—and by association Perkins—has 
been largely ineffective in impacting U.S. 
education and workforce development. 

In this article, I will refer to the mea¬ 
surement of the internal efficiency and 
external effectiveness of CTE as the ROI 
for CTE. This article summarizes the 
contents of the ROI Guidebook for Career and 
Technical Education, which will be pub¬ 


lished by the National Research Center 
for Career and Technical Education 
(NRGCTE) this month. 

The Raw Materials far 
Condiicting ROI: Terminology 

Developing ROI for any organization 
begins with identifying different indica¬ 
tors, usually sorted into inputs, process 
measures, outputs, and outcomes. Inputs 
involve the human, financial and physical 
resources received to support programs, 
activities and services; from the perspec¬ 
tive of GTE programs, examples might 
include funding, enrollments and staffing 
indicators. Process measures include the 
means used to deliver programs, activities 
and services—assessment of student learn¬ 
ing, use of technology and teacher train¬ 
ing all constitute process measures. Outputs 
reflect the quantity of products actually 
produced examples include the number 
of degrees awarded, the number of majors 
in a program, the number of students 
who have transferred to other institu¬ 
tions, or the number of students who have 


graduated. Outcomes cover the quality of 
programs, activities and services—or their 
benefits—to students, states or society. 
Common outcome measures in postsec¬ 
ondary CTE include retention, gradua¬ 
tion and transfer rates; time to degree; test 
scores; and job placements, among others. 

With their emphasis on quantity, inputs 
are easy to count. On the surface, they arc 
easier to assess and understand and gener¬ 
ate less controversy within the education 
community. Process measures may be 
harder to comprehend because they reflect 
non-measurable, qualitative elements in 
an organization. Outcomes are sometimes 
subsumed under the heading of outputs, 
but distinguishing the two as separate 
measures is critical when policy values 
(e.g. y efficiency, equity, choice or qual¬ 
ity) are being discussed. Outputs denote 
quantity and arc therefore mcasurcablc 
and easy to compare. Outcomes repre¬ 
sent policy values, which are much more 
elusive and subjective. Although measur¬ 
ing outcomes is difficult, it is necessary 
because outcomes are the only true mea- 
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FIGURE 1 * ROI terminology: 
How does it oil connect? 


sure of ROI. They are usually expressed 
as questions: Are students learning well? 
Are clients in a training programs getting 
jobs? Are faculty responsive to students? 

Is the institution serving the community 
appropriately? 

ROI usually takes a balance-sheet 
approach, chalking up benefits and costs, 
as shown in f igure 1. Benefits and costs 
themselves arc divided into private [Le. y 
internal to the program) and social (i.e. s 
external to the program) categories. 
Benefits and costs must include the mon¬ 
etised values of all non-monctary benefits 
and costs to fully measure the internal 
efficiency and external effectiveness of a 
program. 

RGIs require the calculation of five 
things: (a) the opportunity cost —a measure 
of what is being given up in order to 
undertake the activity; this measure is 
used to quantify costs and benefits; (b) 
the time horizon —or how long the pro¬ 
gram will be in place, when benefits will 
begin to be observed and fully realized, 
and when costs will begin to appear and 


fully accrue; (c) the discount rate —the 
appropriate rate that needs to be applied 
in order to convert the value of future 
costs and benefits lo the present time; 

(d) monetization- —the translation of all 
non-monctary benefits and costs into 
m on e t a ry va lues; and (e) extern a I ities — 
the measurement of the negative and 
positive impacts of all monetary and 
non-monetary benefits and costs that 
result from having the program in place. 
When any or all of these calculations are 
not undertaken, the result is invariably 
the under- or over-estimation of costs and 
benefits, which in turn skews the following 
measures. 

Broadly speaking, ROI is reflected by 
numbers: these include the benefit cost ratio 
(R/C; a number greater than one implies 
that the program is justified on both inter¬ 
nal efficiency and external effectiveness 
grounds); the net present value (NPV; a num¬ 
ber greater than zero implies that building 
the program today is justified instead of 
waiting for the future); and the internal 
rate of return (IRR; when the rate of return 


obtained from program implementation 
exceeds the market interest rate; ibis is the 
measure used to determine returns from 
financial investing). 

Steps in Conducting ROI: 

An Integrated Logic Model 

There can be as many steps to conduct¬ 
ing ROI analyses as there are terms and 
terminology used to describe them. Here 
are five steps, each addressing a primary 
question: 

■ Needs Assessment: What are some 
of the gaps the program will fill? 

• Feasibility Study: Given certain 
constraints, can the program succeed? 

• Process Evaluation: How is the 
implemented program progressing? 

■ Outcome Evaluation: Were pro¬ 
gram goals and objectives achieved? 

• Cost Analysis: Was the program 
financially worthwhile or valuable? 

As indicated in Figure 2, the logic 
model for ROI involves completing all 
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FIGURE 2* The logic model for measuring, RGL 


five steps and answering each question se¬ 
quentially. Each step focuses on collecting 
the four key indicators—inputs, process 
measures, outputs, and outcomes—from 
which costs and benefits are derived and 
a corresponding success measure is ob¬ 
tained* Underlying the entire logic model 
are scientifically based research methods, 
which when implemented properly gener¬ 
ate optimal ROI analyses. 

ROI for CTE: Applying 
Different Techniques 

ROI is generally conducted in three ways; 

• The common framework tech¬ 
nique must meet the following four 
preconditions: It must include (a) well- 
developed, integrated conceptual 
frameworks; (b) advanced institutional 
research and evaluation expertise; (c) 
highly connected data systems; and 
(d) sound data administration and 
superior management knowledge and 
oversight* This technique requires, at 
a minimum, the explicit linking of 
education and workforce databases 
to measure the impact of a particular 
investment on both direct and in¬ 
direct beneficiaries* 


• The social benefits technique 

measures the total benefits that accrue 
from initiating a spc c i (i c program- 
made action (e.g. , reducing the 
number of CTE dropouts)* The 
programmatic action has both 
direct and indirect impacts* The 
direct impact is quantified as that 
which immediately benefits Lhe 
program’s target population. The 
indirect impact is generally quanti¬ 
fied as a measure of the gain to the 
community* For example, an indirect 
impact could be stated as the follow¬ 
ing: by moving U X V number of 
CTE students from dropout to 
graduation status, a “Y” increase 
in tax collections would result. 
Matching these direct and indirect 
benefits against program costs 
provides a benefit-cost ratio for 
the action. 

• The case study technique identi¬ 
fies Lhe factors that influence the 
success of selected CTE programs, 
and whether such programs are 
achieving a reasonable ROI com¬ 
pared to similar programs* A pro¬ 
gram may be a school, a specific 
occupational program within a 


school, a specific pedagogy, or an 
administrative structure that leads 
to improved student performance* 

The NRCC V E *s R 01G u ideho ok jo r 
Career and Technical Education will present 
examples of the common framework, 
social benefits and case study techniques 
ill at have been conducted at local and 
state levels, 

ROI for CTE: Where We Stand 

Conducting ROI for CTE has been dif¬ 
ficult. Recently, the NRCCTE produced 
an ROI for CTE study that provided a 
primer for conducting ROI under the 
four preconditions required by the com¬ 
mon framework technique (Hollenbeck, 
2011). This study found that participants 
in CTE programs reaped substantial 
returns—positive earnings—with almost 
nil or negative costs associated with sec¬ 
ondary CTE. At the postsecondary level, 
any associated participation costs (?*£*, lu¬ 
ll io n, fo regon e e a r n i ngs) we re more t h a n 
outweighed, even over the short term, by 
the economic payoffs of participating in 
CTE* The common framework technique 
used by Hollenbeck has been applied to 
CTE infrequently because most states and 
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districts find it difficult lo meet all four 
of these preconditions. The field needs 
to find less stringent alternatives without 
sacrificing rigor. 

When the common framework tech¬ 
nique is too difficult lo conduct, a typical 
alternative approach is to cull data on 
benefits and costs from a variety of differ¬ 
ent studies and apply the social benefits 
technique. What distinguishes the social 
benefits from the case study technique 
is a matter of scale: T he social benefits 
technique is generally used when KOI cal¬ 
culations involve both direct and indirect 
impacts, and the latter usually outnum¬ 
bers the former; the case study technique 
focuses more on direct impacts. 

Consider this example* An Alliance 
for Excellent Education study (AEE, 

2010) calculated that about $260,000 per 
dropout would be saved over the student’s 
lifetime if the current dropout rate (30 out 
of 100) were reduced to zero* This study 
also noted that graduating from high 
school generates approximately $10*000 in 
additional annual income. The case study 
technique might focus on this number. 
About 1*3 million high school students 
drop out every year. A EE then estimated 
that the lifetime economic gain reaped by 
eliminating high school dropout would be 
about S335 billion* a number that would 
be provided when applying the social 
benefits technique. 

The NRG GTE has been conducting 
detailed analyses of National Center for 
Education Statistics (NCES) sample sur¬ 
vey data and has developed a typology of 
GTE course-taking (Kotamraju, Aliaga, 
and Dickinson, 2011}. We estimate that 
nearly half of all high school graduates 
Lake at least three or more GTE credits* 
We have also established that high school 
students completing three or more GTE 
Carnegie credits are less likely to drop out 
th an those taking between zero and one 
GTE credits (Aliaga, 2011). That means 
that approximately Slfifi billion of the Iife- 
t i m e ga i n Iro m re dud ng i h e d ro p ou i ra t e 
to zero can be attributed to intensive GTE 


coursetaking* These ROl calculations 
were made possible by using different 
studies that each applied their own unique 
assumptions and specific methodologies. 

In such conditions, care must be taken to 
state and describe these differences clearly, 
addressing any inconsistencies and how 
they have been handled* The NRCCTE 
ROI Guidebook will demonstrate this 
more explicitly. 

Given current policy and budget pres¬ 
sures, GTE needs to advocate for itself and 
develop its own ROI strategy* Otherwise, 
ROI for GTE will be conducted by others, 
with results the field may not like. D 
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